Job Tracking on a Grid – the Logging and Bookkeeping and Job Provenance Services1

Keeping track of a job within a complex Grid environment is a difficult task that cannot be easily delegated to inspection of data from Grid infrastructure monitoring. The job centric monitoring service is used to provide data about actual job status independently on jobs crossing internal administrative domains. It is also a valuable source of data for system administrators helping to improve the infrastructure behavior. The Logging and Bookkeeping (L&B) developed within the EGEE project provides a distributed scalable job centric monitoring service able to deal with hundreds of thousands of jobs on large Grids. To provide the necessary scalability and not to slow down the processing of jobs within a middleware, the service is based on a non-blocking asynchronous model. This means that the order of events sent to L&B by individual parts of the middleware (user interface, scheduler, computing element, etc.) is not guaranteed. A robust on-the-fly processing is used to derive a meaningful job state from events arriving in random order. The L&B may thus temporarily provide information that looks inconsistent with the knowledge user has from some other source (for example, he got an independent notification about the job state). The report provides details of the L&B internal design, and the way of correct interpretation of the L&B results (the job state) is also discussed. While L&B is dealing with active jobs only, the Job Provenance (JP) is designed to permanently store information about all jobs that run on a Grid. All the relevant information, including computing environment specification and basic input data, needed to re-submit the job in the same environment is stored andmade available for a later perusal. Users can annotate stored records, providing yet another metadata layer useful, for example, for job grouping and data mining over the JP. As the data are never